Nice catch!

Should we modify the function _apply_chat_template_to_messages_list to make it compatible with eos_token?

For example:

    def _apply_chat_template_to_messages_list(self, messages_list: InputsType):
        prompts_text = []
        for messages in messages_list:
            InferRequest.remove_response(messages)
            template_inputs, _ = StdTemplateInputs.from_dict({'messages': messages})
            res_context_list, _, _ = self.template._swift_encode(template_inputs)
-           prompts_text.append(''.join(res_context_list))
+           prompts_text.append(''.join(elem for elem in res_context_list if isinstance(elem, str)))

        return prompts_text

@hjh0119 I have read the full self.template._swift_encode(template_inputs) function, the return result is list, if the element of res_context_list is list, this is not corrent result, here should raise an exception instead of ignore the error
so, I think here not judge if isinstance(elem, str), just throw an exception to remind user

if elem type is list, that means the elem is token_id value, not text value.
for example: tokenize.encode("<｜begin▁of▁sentence｜><｜end▁of▁sentence｜>" you will get token_id list, but it is str type befor tokenize.encode().
The elem of res_context_list should all be of type str, and then encoded uniformly in subsequent processing。

I believe the related issue arises because template_meta.prefix contains the BOS token as an integer, which leads to a join failure.

Therefore, I think ignoring integer values in the _apply_chat_template_to_messages_list method is a safer approach, since this method is only used for logging purposes and omitting the BOS token will not have any impact.

Are there any training scripts that fail to run on the main branch but work with this PR?

yes, the related issue arises because template_meta.prefix , I can not agree with you any more. you can see my other pr， this pr solve template_meta.prefix bug。#4813

so are there any training/infer scripts that fail to run on the main branch but work with this PR?

just replace your_own_path in --model, --output_dir, you can test

CUDA_VISIBLE_DEVICES=7 \

swift rollout \

  --model your_own_path/deepseek-ai/deepseek-coder-6.7b-base \

 > logs/log-grpo-rollout.log 2>&1 &



CUDA_VISIBLE_DEVICES=0,1,2,3,4,5 \

NPROC_PER_NODE=6 \

swift rlhf \

  --rlhf_type grpo \

  --model your_own_path/deepseek-ai/deepseek-coder-6.7b-base \

  --reward_funcs accuracy \

  --use_vllm true \

  --vllm_mode server \

  --vllm_server_host 127.0.0.1 \

  --vllm_server_port 8000 \

  --train_type full \

  --torch_dtype bfloat16 \

  --dataset AI-MO/NuminaMath-TIR#1000 \

  --split_dataset_ratio 0 \

  --max_completion_length 512 \

  --num_train_epochs 1 \

  --per_device_train_batch_size 2 \

  --learning_rate 1e-6 \

  --gradient_accumulation_steps 2 \

  --save_total_limit 2 \

  --logging_steps 1 \

  --deepspeed zero2 \

  --max_length 4096 \

  --warmup_ratio 0.05 \

  --dataloader_num_workers 2 \

  --dataset_num_proc 4 \

  --num_generations 6 \

  --temperature 0.9 \

  --top_p 0.9 \

  --top_k 50 \

  --log_completions true \

  --num_iterations 1 \

  --beta 0.01 \

  --output_dir ./ds-base-grpo \

  --report_to tensorboard \

  > logs/log-grpo-test.log 2>&1 &

I've tested the script you provided using your commit, but it still raises the following error: TypeError: sequence item 0: expected str instance, list found.

I've tested the script you provided using your commit, but it still raises the following error: TypeError: sequence item 0: expected str instance, list found.

you should fix #4806 bug first, I have commit pr for this bug